Acerca del set de datos¶

Los datos están relacionados con campañas de marketing directo (llamadas telefónicas) de una institución bancaria portuguesa. El objetivo de la clasificación es predecir si el cliente suscribirá un depósito a plazo (variable y). Los datos están relacionados con campañas de marketing directo de una institución bancaria portuguesa. A menudo, se requería más de un contacto con el mismo cliente, para poder acceder si el producto (depósito bancario a plazo) estaría ('sí') o no ('no') suscrito. Por tanto, el objetivo de la clasificación es predecir si el cliente suscribirá (sí/no) un depósito a plazo (variable y).

In [1]:
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
sns.set_theme(color_codes=True)

import warnings 
warnings.filterwarnings("ignore")
In [2]:
df = pd.read_csv("bank-additional-full.csv", delimiter=";")
In [3]:
pd.set_option("display.max_columns", None)
df.head()
Out[3]:
age job marital education default housing loan contact month day_of_week duration campaign pdays previous poutcome emp.var.rate cons.price.idx cons.conf.idx euribor3m nr.employed y
0 56 housemaid married basic.4y no no no telephone may mon 261 1 999 0 nonexistent 1.1 93.994 -36.4 4.857 5191.0 no
1 57 services married high.school unknown no no telephone may mon 149 1 999 0 nonexistent 1.1 93.994 -36.4 4.857 5191.0 no
2 37 services married high.school no yes no telephone may mon 226 1 999 0 nonexistent 1.1 93.994 -36.4 4.857 5191.0 no
3 40 admin. married basic.6y no no no telephone may mon 151 1 999 0 nonexistent 1.1 93.994 -36.4 4.857 5191.0 no
4 56 services married high.school no no yes telephone may mon 307 1 999 0 nonexistent 1.1 93.994 -36.4 4.857 5191.0 no
In [4]:
df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 41188 entries, 0 to 41187
Data columns (total 21 columns):
 #   Column          Non-Null Count  Dtype  
---  ------          --------------  -----  
 0   age             41188 non-null  int64  
 1   job             41188 non-null  object 
 2   marital         41188 non-null  object 
 3   education       41188 non-null  object 
 4   default         41188 non-null  object 
 5   housing         41188 non-null  object 
 6   loan            41188 non-null  object 
 7   contact         41188 non-null  object 
 8   month           41188 non-null  object 
 9   day_of_week     41188 non-null  object 
 10  duration        41188 non-null  int64  
 11  campaign        41188 non-null  int64  
 12  pdays           41188 non-null  int64  
 13  previous        41188 non-null  int64  
 14  poutcome        41188 non-null  object 
 15  emp.var.rate    41188 non-null  float64
 16  cons.price.idx  41188 non-null  float64
 17  cons.conf.idx   41188 non-null  float64
 18  euribor3m       41188 non-null  float64
 19  nr.employed     41188 non-null  float64
 20  y               41188 non-null  object 
dtypes: float64(5), int64(5), object(11)
memory usage: 6.6+ MB
In [5]:
df.isnull().sum()
Out[5]:
age               0
job               0
marital           0
education         0
default           0
housing           0
loan              0
contact           0
month             0
day_of_week       0
duration          0
campaign          0
pdays             0
previous          0
poutcome          0
emp.var.rate      0
cons.price.idx    0
cons.conf.idx     0
euribor3m         0
nr.employed       0
y                 0
dtype: int64

Análisis exploratorio de los datos¶

In [6]:
#Seleccionar datos categoricos 
df_categoricos = df[["job", "marital", "education", "default", "housing", "loan", "contact", "month", "day_of_week", 
                    "poutcome", "y"]]
df_categoricos.head()
Out[6]:
job marital education default housing loan contact month day_of_week poutcome y
0 housemaid married basic.4y no no no telephone may mon nonexistent no
1 services married high.school unknown no no telephone may mon nonexistent no
2 services married high.school no yes no telephone may mon nonexistent no
3 admin. married basic.6y no no no telephone may mon nonexistent no
4 services married high.school no no yes telephone may mon nonexistent no
In [7]:
#Seleccionar datos númericos
df_numericos = df[["age", "duration", "campaign", "pdays", "previous", "emp.var.rate", "cons.price.idx", 
                  "cons.conf.idx", "euribor3m", "nr.employed"]]
df_numericos.head()
Out[7]:
age duration campaign pdays previous emp.var.rate cons.price.idx cons.conf.idx euribor3m nr.employed
0 56 261 1 999 0 1.1 93.994 -36.4 4.857 5191.0
1 57 149 1 999 0 1.1 93.994 -36.4 4.857 5191.0
2 37 226 1 999 0 1.1 93.994 -36.4 4.857 5191.0
3 40 151 1 999 0 1.1 93.994 -36.4 4.857 5191.0
4 56 307 1 999 0 1.1 93.994 -36.4 4.857 5191.0
In [8]:
# Documentar sns.countplot: https://seaborn.pydata.org/generated/seaborn.countplot.html

cat_vars = ["job", "marital", "education", "default", "housing", "loan", "contact", "month", "day_of_week", "poutcome"]

#Crear figuras con subplots
fig, axs = plt.subplots(nrows=2, ncols=5, figsize = (20, 10))
axs = axs.flatten()

#Crear un countplot para cada variable categorica 
for i, var in enumerate (cat_vars):
    sns.countplot(x=var, hue="y", data = df_categoricos, ax=axs[i])
    axs[i].set_xticklabels(axs[i].get_xticklabels(), rotation = 90)
    
#Ajustar espacio entre subplots
fig.tight_layout()

#Mostrar el plot
plt.show()
In [9]:
# Documentar seaborn.histplot: https://seaborn.pydata.org/generated/seaborn.histplot.html

#Lista  de variables categoricas 
cat_vars = ["job", "marital", "education", "default", "housing", "loan", "contact", "month", "day_of_week", "poutcome"]

#Crear figuras con subplots
fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(20, 10))
axs = axs.flatten()

#Crear histogramas para cada variable categorica 
for i, var in enumerate (cat_vars):
    sns.histplot(x=var, hue="y", data = df_categoricos, ax=axs[i], multiple = "fill", kde = False, element = "bars", fill= True, stat = "density")
    axs[i].set_xticklabels(df_categoricos[var].unique(), rotation=90)
    axs[i].set_xlabel(var)
    
#Ajustar el especio entre subplots
fig.tight_layout()

#Mostrar el plot
plt.show()

  1. La mayoría de las personas que suscriben los depósitos bancarios a plazo son: jubilados y estudiantes.

  2. La mayoría de las personas que suscriben los depósitos bancarios a plazo son cantactados por vía celular.

  3. La mayoría de las personas que suscriben los depósitos bancarios a plazo tienen su último contacto en: octubre, diciembre, marzo, septiembre

  4. La mayoría de las personas que suscriben el depósito bancario a plazo han valorado exitosamente la campaña de marketing.

Resaltar algunos aspectos de la distribución de los datos¶

In [10]:
# Documentar sns.boxplot: https://seaborn.pydata.org/generated/seaborn.boxplot.html

num_vars = ["age", "duration", "campaign", "pdays", "previous", "emp.var.rate", "cons.price.idx", 
                  "cons.conf.idx", "euribor3m", "nr.employed"]

fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(20, 10))
axs = axs.flatten()

for i, var in enumerate(num_vars): 
    sns.boxplot(x=var, data=df, ax=axs[i])
    
fig.tight_layout()
    
plt.show()
In [11]:
# Documentar sns.violinplot: https://seaborn.pydata.org/generated/seaborn.violinplot.html

num_vars = ["age", "duration", "campaign", "pdays", "previous", "emp.var.rate", "cons.price.idx", 
                  "cons.conf.idx", "euribor3m", "nr.employed"]

fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(20, 10))
axs = axs.flatten()

for i, var in enumerate(num_vars): 
    sns.violinplot(x=var, data=df, ax=axs[i])
    
fig.tight_layout()
    
plt.show()
In [12]:
num_vars = ["age", "duration", "campaign", "pdays", "previous", "emp.var.rate", "cons.price.idx", 
                  "cons.conf.idx", "euribor3m", "nr.employed"]

fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(20, 10))
axs = axs.flatten()

for i, var in enumerate(num_vars): 
    sns.violinplot(x=var, y="y", data=df, ax=axs[i])
    
fig.tight_layout()
    
plt.show()
In [13]:
num_vars = ["age", "duration", "campaign", "pdays", "previous", "emp.var.rate", "cons.price.idx", 
                  "cons.conf.idx", "euribor3m", "nr.employed"]

fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(20, 10))
axs = axs.flatten()

for i, var in enumerate(num_vars): 
    sns.histplot(x=var, data=df, ax=axs[i])
    
fig.tight_layout()
    
plt.show()
In [14]:
num_vars = ['age', 'duration', 'campaign', 'pdays', 'previous', 'emp.var.rate', 'cons.price.idx', 
            'cons.conf.idx', 'euribor3m', 'nr.employed']

fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(20, 10))
axs = axs.flatten()

for i, var in enumerate(num_vars):
    sns.histplot(x=var, hue='y', data=df, ax=axs[i], multiple="stack")

fig.tight_layout()

plt.show()
In [15]:
# Documentar sns.pairplot: https://seaborn.pydata.org/generated/seaborn.pairplot.html

# Listado de variables númericas. 
num_vars = ['age', 'duration', 'campaign', 'pdays', 'previous', 'emp.var.rate', 'cons.price.idx', 
            'cons.conf.idx', 'euribor3m', 'nr.employed']

# Crear una Matriz para los diagramas de dispersión
sns.pairplot(df, hue='y')
Out[15]:
<seaborn.axisgrid.PairGrid at 0x1d826033ca0>

Procesamiento de los datos¶

Utilizar pandas.Series.unique¶

Devuelve valores únicos de una serie de objetos.

Los valor únicos se devuelven en orden de aparición. Los valores Únicos se basan en tablas hash, por lo tanto, NO se ordenan.

https://pandas.pydata.org/docs/reference/api/pandas.Series.unique.html

In [16]:
df['job'].unique()
Out[16]:
array(['housemaid', 'services', 'admin.', 'blue-collar', 'technician',
       'retired', 'management', 'unemployed', 'self-employed', 'unknown',
       'entrepreneur', 'student'], dtype=object)
In [17]:
df['marital'].unique()
Out[17]:
array(['married', 'single', 'divorced', 'unknown'], dtype=object)
In [18]:
df['education'].unique()
Out[18]:
array(['basic.4y', 'high.school', 'basic.6y', 'basic.9y',
       'professional.course', 'unknown', 'university.degree',
       'illiterate'], dtype=object)
In [19]:
df['default'].unique()
Out[19]:
array(['no', 'unknown', 'yes'], dtype=object)
In [20]:
df['housing'].unique()
Out[20]:
array(['no', 'yes', 'unknown'], dtype=object)
In [21]:
df['loan'].unique()
Out[21]:
array(['no', 'yes', 'unknown'], dtype=object)
In [22]:
df['contact'].unique()
Out[22]:
array(['telephone', 'cellular'], dtype=object)
In [23]:
df['month'].unique()
Out[23]:
array(['may', 'jun', 'jul', 'aug', 'oct', 'nov', 'dec', 'mar', 'apr',
       'sep'], dtype=object)
In [24]:
df['day_of_week'].unique()
Out[24]:
array(['mon', 'tue', 'wed', 'thu', 'fri'], dtype=object)
In [25]:
df['poutcome'].unique()
Out[25]:
array(['nonexistent', 'failure', 'success'], dtype=object)
In [26]:
df['y'].unique()
Out[26]:
array(['no', 'yes'], dtype=object)

Transformar datos¶

El paquete sklearn.preprocessing proporciona varias funciones comunes que son de utilidad en la transformación de clases para cambiar los vectores de características en una representación que sea más adecuada para los estimadores posteriores.¶

En general, los algoritmos de aprendizaje se benefician de la estandarización del conjunto de datos.¶

https://scikit-learn.org/stable/modules/preprocessing.html

In [27]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['job']= label_encoder.fit_transform(df['job'])
df['job'].unique()
Out[27]:
array([ 3,  7,  0,  1,  9,  5,  4, 10,  6, 11,  2,  8])
In [28]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['marital']= label_encoder.fit_transform(df['marital'])
df['marital'].unique()
Out[28]:
array([1, 2, 0, 3])
In [29]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['education']= label_encoder.fit_transform(df['education'])
df['education'].unique()
Out[29]:
array([0, 3, 1, 2, 5, 7, 6, 4])
In [30]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['default']= label_encoder.fit_transform(df['default'])
df['default'].unique()
Out[30]:
array([0, 1, 2])
In [31]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['housing']= label_encoder.fit_transform(df['housing'])
df['housing'].unique()
Out[31]:
array([0, 2, 1])
In [32]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['loan']= label_encoder.fit_transform(df['loan'])
df['loan'].unique()
Out[32]:
array([0, 2, 1])
In [33]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['contact']= label_encoder.fit_transform(df['contact'])
df['contact'].unique()
Out[33]:
array([1, 0])
In [34]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['month']= label_encoder.fit_transform(df['month'])
df['month'].unique()
Out[34]:
array([6, 4, 3, 1, 8, 7, 2, 5, 0, 9])
In [35]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['day_of_week']= label_encoder.fit_transform(df['day_of_week'])
df['day_of_week'].unique()
Out[35]:
array([1, 3, 4, 2, 0])
In [36]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['poutcome']= label_encoder.fit_transform(df['poutcome'])
df['poutcome'].unique()
Out[36]:
array([1, 0, 2])
In [37]:
from sklearn import preprocessing
label_encoder = preprocessing.LabelEncoder()
df['y']= label_encoder.fit_transform(df['y'])
df['y'].unique()
Out[37]:
array([0, 1])
In [38]:
df.head()
Out[38]:
age job marital education default housing loan contact month day_of_week duration campaign pdays previous poutcome emp.var.rate cons.price.idx cons.conf.idx euribor3m nr.employed y
0 56 3 1 0 0 0 0 1 6 1 261 1 999 0 1 1.1 93.994 -36.4 4.857 5191.0 0
1 57 7 1 3 1 0 0 1 6 1 149 1 999 0 1 1.1 93.994 -36.4 4.857 5191.0 0
2 37 7 1 3 0 2 0 1 6 1 226 1 999 0 1 1.1 93.994 -36.4 4.857 5191.0 0
3 40 0 1 1 0 0 0 1 6 1 151 1 999 0 1 1.1 93.994 -36.4 4.857 5191.0 0
4 56 7 1 3 0 0 2 1 6 1 307 1 999 0 1 1.1 93.994 -36.4 4.857 5191.0 0

Balancear las etiquetas:¶

"Y" Label

Insertar un gráfico de barras para mostrar los conteos de observaciones en cada categeria usando barras.¶

In [39]:
sns.countplot(df['y'])
df['y'].value_counts()
Out[39]:
0    36548
1     4640
Name: y, dtype: int64

Usar función resample https://scikit-learn.org/stable/modules/generated/sklearn.utils.resample.html¶

In [40]:
from sklearn.utils import resample
#Crear dos diferentes dataframe de una clase mayoritaria y minoritaria
df_majority = df[(df['y']==0)] 
df_minority = df[(df['y']==1)] 

# muestreo ascendente de la clase minoritaria
df_minority_upsampled = resample(df_minority, 
                                 replace=True,     # muesta con reemplazo 
                                 n_samples= 36548, # para que coincida con la clase mayoritaria
                                 random_state=0)   # resultados reproducible

# Combinar la clase mayoritaria con la muestra ascendente de la clase minoritaria 
df_upsampled = pd.concat([df_minority_upsampled, df_majority])
In [41]:
sns.countplot(df_upsampled['y'])
df_upsampled['y'].value_counts()
Out[41]:
1    36548
0    36548
Name: y, dtype: int64

Eliminar outliers usando IQR¶

Detectar outlier es tedioso, especialmente cuando se tienen multiples tipos de datos.

Por lo tanto, tenemos diferentes formas de detectar valores atípicos para diferentes tipos de datos.

En cuanto a los datos distribuidos normalmente, podemos obtener el método Z-Score;

Para skewed data, se usa IQR.

IQR es la diferencia entre el cuartil 75th and 25th.¶

In [42]:
def remove_outliers_iqr(df, columns):
    for col in columns:
        q1 = df[col].quantile(0.25)
        q3 = df[col].quantile(0.75)
        iqr = q3 - q1
        lower_bound = q1 - 1.5 * iqr
        upper_bound = q3 + 1.5 * iqr
        df = df[(df[col] >= lower_bound) & (df[col] <= upper_bound)]
    return df

# Señale las columnas para remover los outliers
columns_to_check = ['age', 'duration', 'campaign', 'pdays', 'previous', 'emp.var.rate', 'cons.price.idx', 'cons.conf.idx', 
                    'euribor3m', 'nr.employed']

# Solicitar la función que remueve los outliers usando IQR
df_clean = remove_outliers_iqr(df_upsampled, columns_to_check)

# Mostrar el resultado en el dataframe
df_clean.head()
Out[42]:
age job marital education default housing loan contact month day_of_week duration campaign pdays previous poutcome emp.var.rate cons.price.idx cons.conf.idx euribor3m nr.employed y
37017 25 8 2 7 1 2 0 0 3 3 371 1 999 0 1 -2.9 92.469 -33.6 1.044 5076.2 1
36682 51 9 2 6 0 0 0 0 4 0 657 1 999 0 1 -2.9 92.963 -40.8 1.268 5076.2 1
29384 45 7 2 7 0 0 0 1 0 0 541 1 999 0 1 -1.8 93.075 -47.1 1.405 5099.1 1
21998 29 9 2 3 1 0 0 0 1 4 921 3 999 0 1 1.4 93.444 -36.1 4.964 5228.1 1
16451 37 10 2 2 1 2 2 0 3 4 633 1 999 0 1 1.4 93.918 -42.7 4.963 5228.1 1
In [43]:
df_clean.shape
Out[43]:
(49702, 21)

Correlación mostrando un heatmap¶

Seaborn es una biblioteca de python que permite hacer mejores gráficos fácilmente gracias a su función heatmap(). Un mapa de calor es una representación gráfica de datos donde cada valor de una matriz se representa como un color.

https://seaborn.pydata.org/generated/seaborn.heatmap.html

In [44]:
plt.figure(figsize=(20, 16))
sns.heatmap(df_clean.corr(), fmt='.2g', annot=True)
Out[44]:
<AxesSubplot:>

Definiendo vector de características (X) y variable target (y)¶

In [45]:
X = df_clean.drop('y', axis=1)
y = df_clean['y']

Dividir arrais o matrices en subconjuntos aleatorios de entrenamiento y prueba.¶

https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html

Para ser precisos, el método split() genera los índices de entrenamiento y prueba, no los datos en si mismos.

Tener múltiples divisiones puede ser útil si desea estimar mejor el rendimiento de su modelo.

In [46]:
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.3,random_state=0)

Instanciar el modelo (Decision tree)¶

  1. Instanciamos el modelo con criterio gini index
In [47]:
from sklearn.tree import DecisionTreeClassifier
clf_gini = DecisionTreeClassifier(criterion='gini', max_depth=3, random_state=0)

clf_gini.fit(X_train, y_train)
Out[47]:
DecisionTreeClassifier(max_depth=3, random_state=0)
In [48]:
#Hacemos predicción con CLF Gini
y_pred_gini = clf_gini.predict(X_test)
In [49]:
#Estimamos precisión 
print('Precisión en el set de Entrenamiento: {:.2f}'
     .format(clf_gini.score(X_train, y_train)))
print('Precisión en el set de Test: {:.2f}'
     .format(clf_gini.score(X_test, y_test)))

Precisión en el set de Entrenamiento: 0.85
Precisión en el set de Test: 0.85

Metricas de desempeño¶

Accurracy score con criterio Gini Index

In [50]:
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, jaccard_score
print('F-1 Score : ',(f1_score(y_test, y_pred_gini, average='micro')))
print('Precision Score : ',(precision_score(y_test, y_pred_gini, average='micro')))
print('Recall Score : ',(recall_score(y_test, y_pred_gini, average='micro')))
print('Jaccard Score : ',(jaccard_score(y_test, y_pred_gini, average='micro')))

F-1 Score :  0.8468915565689761
Precision Score :  0.846891556568976
Recall Score :  0.846891556568976
Jaccard Score :  0.7344422472955682
In [51]:
from sklearn.metrics import classification_report, confusion_matrix, roc_curve
print (classification_report(y_test, y_pred_gini))

              precision    recall  f1-score   support

           0       0.87      0.87      0.87      8875
           1       0.81      0.81      0.81      6036

    accuracy                           0.85     14911
   macro avg       0.84      0.84      0.84     14911
weighted avg       0.85      0.85      0.85     14911
In [52]:
# Matriz de confusión
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred_gini)
cm_matrix = pd.DataFrame(data=cm, columns=['Actual Positive:1', 'Actual Negative:0'], 
                                 index=['Predict Positive:1', 'Predict Negative:0'])

sns.heatmap(cm_matrix, annot=True, fmt='d', cmap='Blues')
plt.figure(figsize=(9,9))
Out[52]:
<Figure size 900x900 with 0 Axes>
<Figure size 900x900 with 0 Axes>

Visualización del árbol¶

In [53]:
plt.figure(figsize=(12,8))

from sklearn import tree

tree.plot_tree(clf_gini.fit(X_train, y_train))
Out[53]:
[Text(0.5, 0.875, 'X[10] <= 362.5\ngini = 0.48\nsamples = 34791\nvalue = [20859, 13932]'),
 Text(0.25, 0.625, 'X[18] <= 3.168\ngini = 0.34\nsamples = 22575\nvalue = [17663, 4912]'),
 Text(0.125, 0.375, 'X[19] <= 5087.65\ngini = 0.495\nsamples = 8226\nvalue = [3702, 4524]'),
 Text(0.0625, 0.125, 'gini = 0.352\nsamples = 3618\nvalue = [825, 2793]'),
 Text(0.1875, 0.125, 'gini = 0.469\nsamples = 4608\nvalue = [2877, 1731]'),
 Text(0.375, 0.375, 'X[8] <= 7.5\ngini = 0.053\nsamples = 14349\nvalue = [13961, 388]'),
 Text(0.3125, 0.125, 'gini = 0.03\nsamples = 14167\nvalue = [13949, 218]'),
 Text(0.4375, 0.125, 'gini = 0.123\nsamples = 182\nvalue = [12, 170]'),
 Text(0.75, 0.625, 'X[10] <= 525.5\ngini = 0.386\nsamples = 12216\nvalue = [3196, 9020]'),
 Text(0.625, 0.375, 'X[18] <= 2.916\ngini = 0.492\nsamples = 4384\nvalue = [1913, 2471]'),
 Text(0.5625, 0.125, 'gini = 0.309\nsamples = 2083\nvalue = [398, 1685]'),
 Text(0.6875, 0.125, 'gini = 0.45\nsamples = 2301\nvalue = [1515, 786]'),
 Text(0.875, 0.375, 'X[18] <= 1.402\ngini = 0.274\nsamples = 7832\nvalue = [1283, 6549]'),
 Text(0.8125, 0.125, 'gini = 0.136\nsamples = 2315\nvalue = [170, 2145]'),
 Text(0.9375, 0.125, 'gini = 0.322\nsamples = 5517\nvalue = [1113, 4404]')]

Visualizar el árbol con graphviz¶

In [54]:
#Instalar Graphviz en Python pip install graphviz 
import graphviz 
import pydotplus
%matplotlib inline
In [55]:
dot_data = tree.export_graphviz(clf_gini, out_file=None, max_depth=None,
                                feature_names=X_train.columns,  
                                class_names=True,  
                                filled=True, rotate=True, rounded=True,  
                                special_characters=True)

graph = graphviz.Source(dot_data) 
graph
Out[55]:
Tree 0 duration ≤ 362.5 gini = 0.48 samples = 34791 value = [20859, 13932] class = y 0 1 euribor3m ≤ 3.168 gini = 0.34 samples = 22575 value = [17663, 4912] class = y 0 0->1 True 8 duration ≤ 525.5 gini = 0.386 samples = 12216 value = [3196, 9020] class = y 1 0->8 False 2 nr.employed ≤ 5087.65 gini = 0.495 samples = 8226 value = [3702, 4524] class = y 1 1->2 5 month ≤ 7.5 gini = 0.053 samples = 14349 value = [13961, 388] class = y 0 1->5 3 gini = 0.352 samples = 3618 value = [825, 2793] class = y 1 2->3 4 gini = 0.469 samples = 4608 value = [2877, 1731] class = y 0 2->4 6 gini = 0.03 samples = 14167 value = [13949, 218] class = y 0 5->6 7 gini = 0.123 samples = 182 value = [12, 170] class = y 1 5->7 9 euribor3m ≤ 2.916 gini = 0.492 samples = 4384 value = [1913, 2471] class = y 1 8->9 12 euribor3m ≤ 1.402 gini = 0.274 samples = 7832 value = [1283, 6549] class = y 1 8->12 10 gini = 0.309 samples = 2083 value = [398, 1685] class = y 1 9->10 11 gini = 0.45 samples = 2301 value = [1515, 786] class = y 0 9->11 13 gini = 0.136 samples = 2315 value = [170, 2145] class = y 1 12->13 14 gini = 0.322 samples = 5517 value = [1113, 4404] class = y 1 12->14
  1. Instanciamos el modelo con criterio gini entropy
In [56]:
from sklearn.tree import DecisionTreeClassifier
clf_en = DecisionTreeClassifier(criterion='entropy', max_depth=3, random_state=0)
clf_en.fit(X_train, y_train)
Out[56]:
DecisionTreeClassifier(criterion='entropy', max_depth=3, random_state=0)

Predecir con test set para el criterio entropy

In [57]:
#Hacemos predicciones
y_pred_en = clf_en.predict(X_test)
In [58]:
print('Precisión en el set de Entrenamiento: {:.2f}'
     .format(clf_en.score(X_train, y_train)))
print('Precisión en el set de Test: {:.2f}'
     .format(clf_en.score(X_test, y_test)))

Precisión en el set de Entrenamiento: 0.85
Precisión en el set de Test: 0.85

Metricas de desempeño¶

Accurracy score con criterio Entropy

In [59]:
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, jaccard_score
print('F-1 Score : ',(f1_score(y_test, y_pred_en, average='micro')))
print('Precision Score : ',(precision_score(y_test, y_pred_en, average='micro')))
print('Recall Score : ',(recall_score(y_test, y_pred_en, average='micro')))
print('Jaccard Score : ',(jaccard_score(y_test, y_pred_en, average='micro')))

F-1 Score :  0.8468915565689761
Precision Score :  0.846891556568976
Recall Score :  0.846891556568976
Jaccard Score :  0.7344422472955682
In [60]:
from sklearn.metrics import classification_report, confusion_matrix, roc_curve
print (classification_report(y_test, y_pred_en))

              precision    recall  f1-score   support

           0       0.87      0.87      0.87      8875
           1       0.81      0.81      0.81      6036

    accuracy                           0.85     14911
   macro avg       0.84      0.84      0.84     14911
weighted avg       0.85      0.85      0.85     14911
In [61]:
# Matriz de confusión
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred_en)
cm_matrix = pd.DataFrame(data=cm, columns=['Actual Positive:1', 'Actual Negative:0'], 
                                 index=['Predict Positive:1', 'Predict Negative:0'])

sns.heatmap(cm_matrix, annot=True, fmt='d', cmap='Reds')
plt.figure(figsize=(9,9))
Out[61]:
<Figure size 900x900 with 0 Axes>
<Figure size 900x900 with 0 Axes>
In [62]:
plt.figure(figsize=(12,8))

from sklearn import tree

tree.plot_tree(clf_en.fit(X_train, y_train))
Out[62]:
[Text(0.5, 0.875, 'X[10] <= 362.5\nentropy = 0.971\nsamples = 34791\nvalue = [20859, 13932]'),
 Text(0.25, 0.625, 'X[18] <= 3.168\nentropy = 0.756\nsamples = 22575\nvalue = [17663, 4912]'),
 Text(0.125, 0.375, 'X[19] <= 5087.65\nentropy = 0.993\nsamples = 8226\nvalue = [3702, 4524]'),
 Text(0.0625, 0.125, 'entropy = 0.775\nsamples = 3618\nvalue = [825, 2793]'),
 Text(0.1875, 0.125, 'entropy = 0.955\nsamples = 4608\nvalue = [2877, 1731]'),
 Text(0.375, 0.375, 'X[8] <= 7.5\nentropy = 0.179\nsamples = 14349\nvalue = [13961, 388]'),
 Text(0.3125, 0.125, 'entropy = 0.115\nsamples = 14167\nvalue = [13949, 218]'),
 Text(0.4375, 0.125, 'entropy = 0.351\nsamples = 182\nvalue = [12, 170]'),
 Text(0.75, 0.625, 'X[10] <= 525.5\nentropy = 0.829\nsamples = 12216\nvalue = [3196, 9020]'),
 Text(0.625, 0.375, 'X[18] <= 2.916\nentropy = 0.988\nsamples = 4384\nvalue = [1913, 2471]'),
 Text(0.5625, 0.125, 'entropy = 0.704\nsamples = 2083\nvalue = [398, 1685]'),
 Text(0.6875, 0.125, 'entropy = 0.926\nsamples = 2301\nvalue = [1515, 786]'),
 Text(0.875, 0.375, 'X[18] <= 1.402\nentropy = 0.643\nsamples = 7832\nvalue = [1283, 6549]'),
 Text(0.8125, 0.125, 'entropy = 0.379\nsamples = 2315\nvalue = [170, 2145]'),
 Text(0.9375, 0.125, 'entropy = 0.725\nsamples = 5517\nvalue = [1113, 4404]')]
In [63]:
dot_data = tree.export_graphviz(clf_en, out_file=None, max_depth=None,
                                feature_names=X_train.columns,  
                                class_names=True,  
                                filled=True, rotate=True, rounded=True,  
                                special_characters=True)

graph = graphviz.Source(dot_data) 
graph
Out[63]:
Tree 0 duration ≤ 362.5 entropy = 0.971 samples = 34791 value = [20859, 13932] class = y 0 1 euribor3m ≤ 3.168 entropy = 0.756 samples = 22575 value = [17663, 4912] class = y 0 0->1 True 8 duration ≤ 525.5 entropy = 0.829 samples = 12216 value = [3196, 9020] class = y 1 0->8 False 2 nr.employed ≤ 5087.65 entropy = 0.993 samples = 8226 value = [3702, 4524] class = y 1 1->2 5 month ≤ 7.5 entropy = 0.179 samples = 14349 value = [13961, 388] class = y 0 1->5 3 entropy = 0.775 samples = 3618 value = [825, 2793] class = y 1 2->3 4 entropy = 0.955 samples = 4608 value = [2877, 1731] class = y 0 2->4 6 entropy = 0.115 samples = 14167 value = [13949, 218] class = y 0 5->6 7 entropy = 0.351 samples = 182 value = [12, 170] class = y 1 5->7 9 euribor3m ≤ 2.916 entropy = 0.988 samples = 4384 value = [1913, 2471] class = y 1 8->9 12 euribor3m ≤ 1.402 entropy = 0.643 samples = 7832 value = [1283, 6549] class = y 1 8->12 10 entropy = 0.704 samples = 2083 value = [398, 1685] class = y 1 9->10 11 entropy = 0.926 samples = 2301 value = [1515, 786] class = y 0 9->11 13 entropy = 0.379 samples = 2315 value = [170, 2145] class = y 1 12->13 14 entropy = 0.725 samples = 5517 value = [1113, 4404] class = y 1 12->14

El modelo funciona muy bien tanto con el criterio Gini como Entropy¶

Fin¶